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Abstract 



This paper examines evidence on the “systemic effects” of expanding school choice in Milwaukee, Wisconsin. 
Milwaukee is home to one of the nation’s largest and longest-running school choice programs. If there are 
systemic effects from expanding school choice we should he able to see them in Milwaukee. This paper also 
introduces a novel method for analyzing systemic effects. Taking full advantage of student-level data, we 
develop a new measure of those effects based on the extent of voucher options that each student has each year. 
The idea behind this measure is that school systems face greater competitive pressure to serve students well 
when students have more options to leave. This type of measure might be useful for future analyses of systemic 
effects. Using this new approach, we find that students fare better academically when they have more options 
from Milwaukee’s voucher program. The effects are modest in magnitude, but they are robust to multiple 
specifications of the model. 



Introduction 

When choice and competition in education are expanded by offering families vouchers, how is the achievement 
of students who remain in traditional public schools affected? This is really the central policy question 
surrounding school choice. If choice works as its promoters have suggested, then student achievement in 
traditional public schools should rise as those schools experience greater motivation to attract students and 
the revenue those students generate. But if choice works as its detractors suggest, student achievement in 
traditional public schools should decline as talent and resources are drained from those public schools, making 
it harder for them to teach effectively. 

Of course, one could support school choice even if it produced no improvement in traditional public schools as 
a “lifeboat” for individual students faring poorly in their prior public schools. But given the limited number of 
students likely to participate in even the largest school choice programs, the biggest policy potential (or peril) 
for school choice is in its effects on the whole school system, not in its effects on participating students. 

This paper examines evidence on the “systemic effects” of expanding school choice in Milwaukee, Wisconsin. 
Milwaukee is home to one of the nation’s largest and longest-running school choice programs. If there are 
systemic effects from expanding school choice we should be able to see them in Milwaukee. 

This paper also introduces a novel method for analyzing systemic effects. Taking full advantage of student 
level data, we develop a new measure of those effects based on the extent of voucher options that each student 
has each year. The idea behind this measure is that school systems face greater competitive pressure to serve 
students well when students have more options to leave. This type of measure might be useful for future 
analyses of systemic effects. 

Using this new approach, we find that students fare better academically when they have more options 
from Milwaukee’s voucher program. The effects are modest in magnitude, but they are robust to multiple 
specifications of the model. 
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This report and its companion reports are the second in a series of annual reports on the Milwaukee Parental 
Choice Program (MPCP) that will he conducted hy the School Choice Demonstration Project (SCDP). Future 
reports on the systemic effects of the program will examine such topics as the supply-side response to school 
choice expansion and the effect of choice on integration hy race and ethnicity in schools. 

An initial draft of this report was greatly improved based on comments from the SCDP Research Advisory 
Board, particularly David Camphell of the University of Notre Dame and Tom Loveless of the Brookings 
Institution. This ongoing research project is being funded by a diverse set of philanthropies including the Annie 
E. Casey, Joyce, Kern Family, Lynde and Harry Bradley, Robertson, and Walton Family Foundations. We thank 
them for their generous support and acknowledge that the actual content of this report is solely the responsibility 
of the authors and does not necessarily reflect any official positions of the various funding organizations or the 
University of Arkansas. Finally, we are grateful to the Milwaukee Public Schools, the private schools in the 
MPCP, and the state Department of Public Instruction for the willing cooperation, advice, and assistance that 
has made this research possible. 

Prior Research 

There have been a fairly large number of high quality studies on the effects of private school choice on the 
academic achievement of participating students. Of the ten studies with a random-assignment research design, 
all but one show significant positive effects for participating students for at least some sub-groups of students in 
some subjects. But, as we mentioned, the bigger policy question revolves around how expanding school choice 
with vouchers affects academic achievement for students who remain in traditional public schools. 

On the question of systemic effects from expanding school choice with vouchers, rigorous evidence is less 
common than for participant effects. We have relatively few large and long-running programs to examine. 

And, more importantly, designing a rigorous study is complicated by the difficulty in identifying an appropriate 
counter-factual. That is, it is very hard to know how students would have fared in school systems had there not 
been an expansion of choice and competition. We can compare growth in student achievement in districts with 
greater choice and competition to achievement in districts with less choice and competition, but no two districts 
are identical. It is always hard to know whether greater achievement across districts is due to the difference in 
choice and competition or to some other, unobserved differences between those districts. 

A fairly large literature exists examining the effects of choice and competition among school operators without 
the introduction of voucher programs by focusing on competition between school districts or between public 
and private schools. Clive Belfleld and Henry Levin have reviewed that literature and conclude: “The sampling 
strategy identified more than 41 relevant empirical studies. A sizable majority report beneficial effects of 
competition, and many report statistically significant correlations. For each study, the effect size of an increase 
of competition by one standard deviation is reported. The positive gains from competition are modest in 
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scope with respect to realistic changes in levels of competition. The review also notes several methodological 
challenges and recommends caution in reasoning from point estimates to public policy.”' 

That final sentence is worthy of elaboration. Competition between school districts or between already existing 
public and non-subsidized private schools may be fundamentally different in its effects than competition 
produced by the introduction of a school choice program. While Belfield and Levin’s review reminds us that 

competition in education exists 
even in the absence of school 
choice programs, and gives 
us some reason to expect that 
competition should be positively 
related to achievement, we cannot 
draw strong conclusions about 
the likely effects of voucher 
programs on public school student 
achievement from that literature. 

But there are also a number of 
studies that have specifically 
examined the effect of expanded 
choice and competition from 
voucher programs. Programs 
in Florida and Milwaukee have 
been the subject of the most 
examination. In Florida there have 
been four studies conducted on the 
effects of the Opportunity Scholarship voucher program that was part of the state’s A+ accountability system.^ 
Under that program, schools were graded from A to F based on a number of student achievement indicators. 
Schools that received two failing grades in a four-year period had to offer vouchers to their students with which 
those students could leave for other public or private schools. 

All four studies found that schools made exceptionally large improvements in student achievement when faced 
with the prospect of receiving a second failing grade and having to offer vouchers to their students. The studies 




1 Clive R. Belfield and Henry M. Levin, "The Effects of Competition Between Schools on Educational Outcomes: A Review for the 
United States',' Review of Educational Research, Vo\. 72, No. 2, 279-341 (2002). 

2 Rajashri Chakrabarti, "Vouchers, Public School Response and the Role of Incentives: Evidence from Florida," Federal Reserve 
Bank of New York Staff Report, Number 306, October 2007; Jay P. Greene and Marcus A. Winters, "Competition Passes the 
Jestf Education Next, Summer 2004; Cecilia Elena Rouse, Jane Hannaway, Dan Goldhaber, and David Figlio, "Feeling the Heat: 
How Low Performing Schools Respond to Voucher and Accountability Pressure,"CALDER Working Paper 13, Urban Institute, 
November 2007; Martin West and Paul Peterson, "The Efficacy of Choice Threats Within School Accountability Systems," 
Harvard PEPG Working Paper 05-01, March 23, 2005 (subsequently published in The Economic Journal, March, 2006). 
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differ in the extent to which they attribute these gains to the competitive effect of the vouchers or the stigma of 
receiving a failing grade. West and Peterson, as well as Rouse, et ah, cannot rule out the stigma explanation. 

But Chakraharti as well as Greene and Winters empirically examine and fail to find a strong stigma effect. 
Chakraharti does so hy comparing the effects of schools receiving a failing grade in Florida before the voucher 
provision was introduced to the effects after the voucher component was added and finds a significantly 
larger improvement after vouchers were included. Greene and Winters explore the possible stigma effect by 
examining how failing schools fared once the four-year “window” for being eligible for vouchers had closed. 
They found that failing schools significantly slipped once the possibility of having vouchers had been removed 
by receiving three subsequent passing grades. Neither test for stigma is definitive, but the Florida evidence 
gives us some reason to believe that the threat 
of voucher competition spurs significant 
improvement in traditional public schools. 

There is also another systemic effects study in 
Florida by Greene and Winters, but it examines 
the McKay Scholarship voucher program for 
disabled students rather than the Opportunity 
Scholarship.^ The McKay program offers 
vouchers to all disabled students in Florida 
public schools with which they can attend 
other public or registered private schools. The 
question examined by Greene and Winters was 
whether disabled students remaining in Florida 
public schools performed better academically when their school faced greater competitive pressure from nearby 
private schools receiving McKay Scholarship students. The measure of competition their study employs is the 
number of private schools registered to participate in the McKay program within a five or ten mile radius of 
each public school. The study found that disabled students performed significantly better when their school 
experienced an increase in competitive pressure from the McKay program. 

Three rigorous studies have also been conducted on the systemic effects of the voucher program in Milwaukee.'* 
Both Hoxby and Chakraharti find that schools exposed to greater voucher competition make greater academic 
improvement than schools in Milwaukee less exposed, and greater improvement than demographic ally 
similar schools outside of Milwaukee unexposed to competition from the voucher program. The difficulty 




3 Jay P. Greene and Marcus A. Winters, "The Effect of Special Education Vouchers on Public School Achievement: Evidence From 
Florida's McKay Scholarship Program," Manhattan Institute, Civic Report Number 52, April 2008. 

4 Martin Carnoy, et al. "Vouchers and Public School Performance," Economic Policy Institute, October 2007; Rajashri Chakraharti, 
"Can Increasing Private School Participation and Monetary Loss in a Voucher Program Affect Public School Performance? 
Evidence from Milwaukee," Federal Reserve Bank of New York, 2007 (forthcoming in the Journal of Public Economics;) Caroline 
Minter Hoxby, "The Rising Tide," £ducflt/on A/exf, Winter 2001. 
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is in measuring how exposed each school is to voucher competitive pressure. Hoxhy and Chakraharti use 
the percentage of students eligible for subsidized lunch and therefore meeting the income requirements for 
eligibility for the voucher program as a proxy for competitive pressure. Schools with more students eligible to 
leave with vouchers were classified as more exposed to voucher competition. 

Camoy, et al. successfully replicate these findings, but they conduct additional analyses with alternative 
measures of competition that fail to show positive results. In particular, they employ a measure similar to the 
one used by Greene and Winters in their evaluation of the McKay Scholarship program whereby the extent of 
competitive pressure is measured by the proximity to each public school of voucher-receiving private schools. 
Using that measure of competition Camoy, et al. fail to find any competitive effect. Of course, it is also possible 
that proximity in Milwaukee is not a particularly good way to measure competitive pressure given the city’s 
small size and accessible public transportation system, but the lack of robustness across different measures 
undermines confidence in positive results. 

Camoy, et al. also extend the analysis conducted by Hoxhy and Chakraharti to subsequent years and fail to find 
continued improvement in those later years. That is, using Hoxhy and Chakrabarti’s measure of competition 
Camoy, et al. find that schools more exposed to competitive pressure improved after the program expanded 
in 1998 but those improvements did not grow larger in later years. Of course, it could be that the gains from 
competition had already occurred and did not expand without an additional significant expansion in the size of 
the program. But again, the lack of compounding gains undermines confidence in Hoxhy and Chakrabarti’s 
positive results. 

This study builds upon the work by Hoxhy, Chakraharti, and Carnoy, et al. The present study differs in that it 
has individual student data while the previous studies relied upon aggregate school results. And because we 
have individual student data we are able to employ a novel measure that captures the extent of competition for 
each student. These new data and new approach may help resolve some of the issues raised in prior research in 
Milwaukee and Florida. 

Data and Method 

There were two sets of data used, voucher-using student data and public school student achievement data. The 
first dataset contained student-level data on students enrolled in the Milwaukee Parental Choice Project (MPCP) 
voucher program. These data included student addresses and the name of the voucher-accepting school that the 
student attended. 

The MPCP dataset was used in the creation of the voucher-options variable - our novel measure of the options 
available to each student in each year. The student addresses were geocoded and compared to a geocoded set 
of the schools in the Milwaukee Public School district to determine the nearest public school to that student. 

This school was then paired with the actual voucher-accepting school attended by the student to determine the 
distance between the two schools. From this distance measure four values — the 25* percentile (.940 miles), the 
median (2.02 miles), the 75* percentile (3.68 miles) and the 90* percentile (5.56 miles) — were chosen to act as 
a radii for the voucher-options variable. We also use as a fifth “radius” the whole city of Milwaukee. This allows 
us to estimate our chosen model with varying radii as a robustness check. 
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To repeat, we cannot actually observe the distance between where MPS students live and voucher-accepting 
private schools because we do not have the addresses of MPS students. But we can measure the distance 
between each MPS student’s school and voucher-accepting options. We understand that using the distance from 
the MPS school is an imperfect proxy for the distance from each student’s home, but given the data limitations 
this is the best method for gauging the proximity of MPS students to their voucher options. 

For an observation on a student in a given year, the voucher-options variables are interactions between a free- 
lunch status indicator variable and counts of private schools meeting the following requirements for that year: 

1) are located closer to the MPS school than the radius of interest; 2) serve the grade in which the student is 
enrolled; and 3) accepted vouchers during the year observed. Free or reduced-price lunch status was used to 
proxy for voucher eligibility; because of the structure of the voucher program, we can be certain that students 
eligible for a free or reduced-price lunch are eligible for a voucher. That is, the number of voucher options 
available to each student is a function of their eligibility for the program (free lunch status) and the number 
of voucher-receiving private schools in given grade for a given year within a given distance. The number of 
options increases or decreases for each student if their eligibility for the program changes or if the number of 
private schools serving the next grade-level changes or if more private schools come on-line over time. 

For an example of the voucher-options creation process, we can examine a hypothetical student and construct 
her voucher-options count. Consider a 3“^ grader who was eligible for free or reduced-price lunch in the 1 999- 
2000 school year. Her median-radius voucher-options value is a count of the number of voucher-accepting 
schools that were open for the 1999-2000 school year, that served the 3“* grade, and that were located closer to 
her attended MPS school than 2.01 miles. However, if the third grader was not eligible for free or reduced-price 
lunch, the voucher-options value for the student would be zero. Descriptive and summary information for the 
voucher-options variables can be seen in Table 1 . 

The second set of data contained public school student achievement information. The achievement data 
obtained were a panel of individual data of the universe of students from the Milwaukee Public School (MPS) 
system from 1999 through 2006.^ The data included grade, year, school attended, testing information, free 
and reduced-price lunch status, and a unique identifier that allowed us to follow each student through time. 
Descriptive and summary information on these variables can be seen in Table 1 . 

Student achievement data in our panel comes from two tests, the Wisconsin Knowledge Concepts Examination 
(WKCE) and the Terra Nova Achievement Examination (Terra Nova). Both tests examine knowledge in 
language arts, mathematics, and reading, while the WKCE additionally examines social studies and science 
knowledge. 

The testing policy regime for Milwaukee changed during the time period covered by the panel. From 1 999 until 
2006, students in grades 4, 8, and 10 were given all five components of the WKCE. Students in other grades 
varied from year to year on what test, if any, they received. The tests taken by each grade in each year can be 
seen in Table 2. 



5 Years refer to the first year of an academic year; for example, 1999 refers to the 1999-2000 school year. 
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The test scores in the MPS student achievement dataset are measured in normal curve equivalent (NCE) units. 
NCEs place the student on a normal curve defined by a representative sample of students taking the test. NCEs 
are similar to percentile ranks in that they allow comparison of achievement across grade but they are favorable 
in that they are equal-interval scaled. The equal-interval scale means that the difference between two scores 
measures a gap of the same magnitude as that between two other scores with the same difference. Eor instance, 
the gap between scores of 45 and 50 is of the same magnitude as the gap between 50 and 55. 

We run a student-level fixed-effects analysis comparing test scores to the voucher-options count and use the 
fixed effects parameters to control for unobserved, time-invariant characteristics of the students: 

(j) Y.^ = Voucher Options -l-OZ +a^ +s.^ 

where Z is a sometimes-included vector of controls for test, grade and/or year; a. are the student fixed-effects 
parameters; and ^u, the error term, is assumed to cluster around school grade level in order to account for the 
shared experience of students enrolled in a particular grade level in a particular school. 

Some students took different tests (WKCE and Terra Nova) in different years. To account for the possibility that 
the tests might not be comparable even when using normal curve equivalents, we estimate the model over three 
different samples: the full sample, observations that come from the WKCE, and observations that come from the 
Terra Nova. When the model is run over the full sample, we include a control for the test that was taken.'^ 

The number of voucher-accepting schools may not be the only achievement-affecting change between years and 
grades. To account for anything else that may have changed between years and grades and might have affected 
student outcomes, we run the model with 4 separate specifications. We run it with no controls, with controls for 
year, with controls for grade, and with controls for both year and grade. 

Conceptually, our model captures whether student achievement is higher or lower when each student has more 
or fewer voucher options available to him or her. The student fixed effect should control for all time-invariant 
characteristics of each student. Essentially, we are comparing students against themselves. If Milwaukee 
Public Schools are competing effectively, they should serve students better as those students have more options 
available to them to leave the public system and student achievement should rise. 

It may be difficult to visualize exactly how MPS more effectively serves students with more voucher options 
given the difficulty of school officials to make fine distinctions between what is provided to each student. 

But school officials do not have to make fine distinctions for them to be more attentive to students with more 
options. Students with more options tend to be clustered in particular grades, in particular schools, and in 
particular years. MPS simply has to react to the fact that the entry of a few voucher middle schools, for 
example, creates more pressure to serve students in middle school grades - especially in schools with higher 
concentrations of subsidized lunch students who are eligible for vouchers. MPS officials may be able to detect 



6 We also ran analyses in which we converted both WKCE and Terra Nova scores to "z-scores" based on the within sample mean 
and standard deviation. Converting to z-scores produces similar results. 
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the schools, grades, and times when they face greater competitive challenges without being conscious of that 
competition or discussing it with others. 

We also considered a model in which the dependent variahle would he the gain in student achievement from the 
previous year’s test result. A potential advantage of such a model is that 
it would capture any trend in a student’s achievement. Model (1) above 
only compares student achievement in each year to the student’s average 
achievement, as captured in the student fixed-effect. Unfortunately, 
the alternative model would discard roughly half of the available data 
because it would require that students have test results in the same subject 
in subsequent years. Given the irregular testing schedule (see Table 
2) and “holes” in the MPS records, we would lose roughly half of our 
observations. And data would be excluded in a non-random way that 
might bias any findings. In addition, we believe that controlling for year 
effectively “de-trends” test results without a comparable loss in data. 

Nevertheless, we performed analyses with the change in test score as 
the dependent variable. The results are generally similar to those of our 
preferred model with selected results presented below. Detailed results of 
this alternative model are available from the authors upon request. 

Results 

Results from the initial combined-sample estimations can be seen in Table 
[A]. The coefficients for the voucher-options variable are positive and 
highly significant across subjects and radii. 

The reported coefficients measure the expected change in NCE score from the addition of one voucher option 
inside the specified radius from the school attended by an MPS student. For instance, Table [A] shows that the 
addition of one voucher-accepting school anywhere in Milwaukee will improve student achievement on the 
Language Arts portion of either the WKCE or the Terra Nova by .055 NCE points. The addition of a voucher- 
receiving private school would increase a student’s standardized Math score by .047 NCE and Reading score by 
.058 NCE. 

If we had instead reported the analyses using the gain in test score as the dependent variable, the impact of 
adding a voucher-accepting private school would have been a gain of .178 NCE in Language Arts, .107 NCE in 
Mathematics, and .148 in Reading. All three results are statistically significant at p < .01. But as stated above, 
we do not prefer this gain-score model because it discards a large number of observations in a non-random 
way that could distort results. The point is that this alternative specification produces results that are generally 
consistent with the main results we report. 

Another way to measure the effect is to examine the effect size in terms of standard deviations. Using the 
coefficient for Language Arts, for example, a one standard deviation change in the number of voucher-accepting 
schools in Milwaukee (36.96 schools) would lead to a test score gain of 2.03 NCE points. NCE scales are set to 
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have a standard deviation of 21 . 06 , so the one standard deviation change in voucher options equates to a change 
of . 1 0 standard deviations in NCE score. Effect sizes for the models using a whole-city voucher-options count 
and no controls are shown in Table [E] 

This method was used to calculate effect sizes for all of the regressions. Although 
the coefficients drop steadily as the radius is expanded, the effect sizes are roughly 
constant across choice of radius. The same competitive effects of the voucher 
program are being measured in each regression, but the effects are divided between 
fewer schools when the radius is small. This suggests that MPS schools feel 
competition from any voucher school opened in Milwaukee, not just the schools 
opened nearby. 

The results for the separated sample regressions can be seen in Table [B]. While 
all coefficients were positive and significant, it is worth noting that the values on 
analyses of the WKCE are consistently greater than those on the Terra Nova. This 
may provide evidence that the WKCE is emphasized more than the Terra Nova in 
MPS schools, but also may be a result of the changes through time to the WKCE 
and the creation of its NCE values. That being said, however, the coefficients for the 
Terra Nova are positive and significant. The results on both sub-samples reinforce the 
combined sample result that MPS students are improving as the number of voucher- 
accepting school options for them increases. 

Results for the separated sample regressions with various controls can be seen in 
Tables [C]-[E]. Though the values for smaller radii are inconsistent, the regressions 
using the whole city count are overwhelmingly positive and significant. Since Table 
[A] shows that the addition of a voucher-accepting school anywhere in Milwaukee 
has an effect on MPS schools, the whole-city count is preferred to the smaller radii 
counts. The whole city count models show that even after controlling for test taken, for year, and for grade, the 
number of voucher-accepting school options has a positive relationship with student achievement. 

Discussion and Conclusion 

Using an individual fixed-effects model with a novel measure of systemic effects, we find that students in 
Milwaukee fare better academically when they have more free private options through the voucher program. It 
appears that Milwaukee public schools are more attentive to the academic needs of students when those students 
have more opportunities to leave those schools. This finding is robust across several different specifications of 
the model. 

This finding is also consistent with most prior studies of systemic effects from voucher programs in Milwaukee 
and Elorida. In addition, this study may help explain the null results Camoy, et al. produced using a proximity 
measure of competition in Milwaukee. We did not find a greater competitive response from public schools to 
private options located closer to them than to private options located elsewhere in the city. If that is correct, 
then proximity is not a useful way to capture variation in competitive pressure. So, our results are also 
consistent with Camoy, et al.’s results. 
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While we find positive results from voucher competition in Milwaukee, it is also important to put in perspective 
the magnitude of the benefits. A one standard deviation increase in private options available to each student 
results in about one-tenth of a standard deviation increase in achievement. Results of that magnitude suggest 
that voucher competition has been a positive if not a transforming force in the Milwaukee public school system. 
If 37 additional private schools participate in the MPCP, we can expect that student achievement for students 
remaining in MPS will rise by about one-tenth of a standard deviation or 2 NCE points on standardized tests. 

We come to this conclusion by observing the variation in voucher options available to students over the last 
few years. But if we can extrapolate our findings to the entire history of MPCP, then the effects of choice and 
competition on MPS students is less modest. Over the last two decades Milwaukee has gone from having no 
voucher-accepting schools to having 124 as of 2007-08. If each addition of 37 voucher schools increases MPS 
performance by 2 NCE, then the existence 
of MPCP over the last two decades has 
improved MPS student achievement by 
6.7 NCE on standardized tests. Of course, 
we cannot generalize outside of the period 
we observed with confidence, but it helps 
consider the overall effect of having the 
voucher program and not just the effect of 
recent expansion. 

The best way to increase confidence in 
these findings is to continue studying 
the effects of choice and competition in 
Milwaukee and elsewhere. No single 
study of a single program can be definitive, 
but this research contributes to the finding 
that expanded choice and competition 
improve the academic performance of 
students who remain in traditional public schools. 
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Table 1 : Descriptive statistics 



Variable 


Descrintion 


Obs 


Mean 


Std. 

Dev. 


Min 


Max 


vou25 


Number of voucher accepting schools 
within .94 miles of attended public school 


370516 


2.5804 


2.1376 


0 


10 


vou50 


Number of voucher accepting schools 
within 2.02 miles of attended public school 


370516 


10.154 


6.8782 


0 


31 


vou75 


Number of voucher accepting schools 
within 3.68 miles of attended public school 


370516 


25.526 


15.113 


0 


62 


vou90 


Number of voucher accepting schools 
within 5.56 miles of attended public school 


370516 


42.552 


21.442 


0 


84 


WCMP 


Number of voucher accepting schools 
within the city of Milwaukee 


370516 


65.928 


27.024 


9 


98 


freelunch 


Indicator of eligibility for free or 
reduced-price lunch 


370784 


0.7485 


0.4339 


0 


1 


voucomp25 


Free-lunch-weighted voucher options count 
within .94 miles of attended public school 


370516 


2.0767 


2.2427 


0 


10 


voucompSO 


Free-lunch-weighted voucher options count 
within 2.02 miles of attended public school 


370516 


8.0873 


7.6156 


0 


31 


voucomp75 


Free-lunch-weighted voucher options count 
within 3.68 miles of attended public school 


370516 


20.268 


17.59 


0 


62 


voucomp90 


Free-lunch-weighted voucher options count 
within 5.56 miles of attended public school 


370516 


33.441 


26.66 


0 


84 


voucompWC 


Free-lunch-weighted voucher options count 
in the city of Milwaukee 


370516 


50.931 


36.965 


0 


98 


TN 


Indicator that test observation is from the 
Terra Nova 


370782 


0.4857 


0.4998 


0 


1 


nee Language 
Arts 


NCE score on language arts examination 


243660 


45.029 


19.102 


1 


99 


nceMathematics 


NCE score on mathematics examination 


310876 


40.363 


19.321 


1 


99 


nceReading 


NCE score on reading examination 


291492 


42.548 


19.482 


1 


99 


nceScience 


NCE score on WKCE science examination 


84337 


39.5 


19.231 


1 


99 


nceSocial 

Sciences 


NCE score on WKCE social sciences 
examination 


83920 


41.803 


19.291 


1 


99 
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Table 2 : Test observations in Milwaukee 



\Year 

Grad^ 


1999 


2000 


2001 


2002 


2003 


2004 


2005 


2006 


2 


- 


TN:aU 


- 


- 


- 


- 


- 


- 


3 


- 


TN:aU 


TN:mi 


TN:m 


TN:aU 


TN:aU 


TN: m, 


TN: m, r 


4 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


5 


- 


TN:aU 


TN:aU 


TN:aU 


TN: aU 


TN:aU 


TN: m,r 


TN: m, r 


6 


- 


TN:aU 


TN:aU 


TN:aU 


TN:aU 


TN:aU 


TN: m, r 


TN: m,r 


7 


- 


TN:aU 


TN:aU 


TN:aU 


TN: aU 


TN:aU 


TN: m, r 


TN: m, r 


8 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


9 


- 


TN:aU 


TN:aU 


TN:aU 


TN:aU 


TN:aU 


TN:aU 


- 


10 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE: aU 


WKCE:aU 


WKCE: aU 


WKCE: aU 



1. Third graders in 2001 and 2002 received oniy the mathematics portions of the Terra Nova. 

2. in 2005 and 2006, only the mathematics and reading portions of the Terra Nova were administered. 
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Table [A]: Combined sample regressions, voucher-options coefficients only 







25 


50 


75 


90 


Whole 


Observatiom 






(.94 m) 


(2.02 m) 


(3.68 m) 


(5.56 m) 


City 


( Groups) 


No 


Language Arts 


0.513*“ 


0.235*** 


0.105*** 


0.075*** 


0.055*** 


243491 


Controls 




(0.090) 


(0.031) 


(0.013) 


(0.008) 


(0.006) 


(97598) 




Mathematics 


0.466*“ 


0.190*** 


0.086*** 


0.062*** 


0.047*** 


310608 






(0.100) 


(0.031) 


(0.013) 


(0.008) 


(0.005) 


(112146) 




Reading 


0.567*** 


0.244*** 


0.110*** 


0.077*** 


0.058*** 


291226 






(0.092) 


(0.031) 


(0.013) 


(0.008) 


(0.006) 


(109637) 




’within-R? 


0.07 


0.07 


0.07 


0.07 


0.07 






OLSR^ 


0.80 


0.80 


0.80 


0.80 


0.80 




Year 


Language Arts 


0.160*** 


0.106*** 


0.041*** 


0.030*** 


0.028*** 


243491 






(0.059) 


(0.021) 


(0.008) 


(0.005) 


(0.003) 


(97598) 




Mathematics 


-0.096 


-0.016 


-0.016* 


-0.009* 


-0.004 


310608 






(0.072) 


(0.020) 


(0.008) 


(0.005) 


(0.003) 


(112146) 




Reading 


0.110* 


0.078*** 


0.029*** 


0.020*** 


0.020*** 


291226 






(0.060) 


(0.021) 


(0.008) 


(0.005) 


(0.004) 


(109637) 




nuithin-E? 


0.16 


0.16 


0.16 


0.16 


0.16 






OLSR^ 


0.82 


0.82 


0.82 


0.82 


0.82 




Grade 


Language Arts 


0.025 


0.056“ 


0.016“ 


0.012*** 


0.003 


243491 






(0.058) 


(0.023) 


(0.007) 


(0.004) 


(0.002) 


(97598) 




Mathematics 


-0.082 


0.002 


-0.003 


0.001 


0.003 


310608 






(0.061) 


(0.019) 


(0.007) 


(0.004) 


(0.002) 


(112146) 




Reading 


-0.010 


0.037“ 


0.008 


0.005 


0.000 


291226 






(0.052) 


(0.019) 


(0.006) 


(0.004) 


(0.002) 


(109637) 




nuithin-R? 


0.15 


0.15 


0.15 


0.15 


0.15 






OLSR^ 


0.81 


0.81 


0.81 


0.81 


0.81 




Year and 


Language Arts 


0.006 


0.046“ 


0.009 


0.008* 


0.010*** 


243491 


Grade 




(0.055) 


(0.023) 


(0.007) 


(0.004) 


(0.002) 


(97598) 




Mathematics 


-0.069 


0.006 


-0.003 


0.002 


0.009*** 


310608 






(0.060) 


(0.018) 


(0.007) 


(0.004) 


(0.002) 


(112146) 




Reading 


-0.022 


0.028 


0.003 


0.002 


0.005*** 


291226 






(0.050) 


(0.019) 


(0.006) 


(0.004) 


(0.002) 


(109637) 




nuithin-R? 


0.18 


0.18 


0.18 


0.18 


0.18 






OLSR^ 


0.82 


0.82 


0.82 


0.82 


0.82 





Robust standard errors in parentheses. Errors clustered around school grade level. 
* Significant at 10% confidence level 

** Significant at 5% confidence level 

*** Significant at 1% confidence level 
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Table [B]: Separated samples, no controls, voucher-options coefficients oniy 







25 

(.94 m) 


50 

(2.02 m) 


75 

(3.68 m) 


90 

(5.56 m) 


Whole 

City 


Observations 
( Groups) 


WKCE 


Language Arts 


1.305*** 


0.566*** 


0.267*** 


0.185*** 


0.132*** 


84070 






(0.374) 


(0.112) 


(0.043) 


(0.026) 


(0.018) 


(68444) 




Mathematics 


1.065*** 


0.438*** 


0.214*** 


0.151*** 


0.108*** 


132781 






(0.261) 


(0.081) 


(0.034) 


(0.022) 


(0.015) 


(90961 




Reading 


0.894*** 


0.373*** 


0.181*** 


0 126*** 


0.089*** 


131814 






(0.212) 


(0.069) 


(0.028) 


(0.019) 


(0.012) 


(90155) 




Social Studies 


\ 974*** 


0.838*** 


0.405*** 


0.284*** 


0.217*** 


83750 






(0.515) 


(0.139) 


(0.056) 


(0.035) 


(0.023) 


(68139) 




Science 


1.936*** 


Q 


0.394*** 


0.279*** 


0.220*** 


84169 






(0.465) 


(0.135) 


(0.055) 


(0.034) 


(0.024) 


(68446) 




within-R^ 


0.03 


0.03 


0.03 


0.03 


0.03 






OLSR^ 


0.91 


0.91 


0.91 


0.91 


0.91 




Terra 


Language Arts 


0.111* 


0.090*** 


0.034*** 


0.024*** 


0.018*** 


159491 


Nova 




(0.065) 


(0.024) 


(0.008) 


(0.005) 


(0.004) 


(75711) 




Mathematics 


0.147* 


0.077*** 


0.027*** 


0.022*** 


0.017*** 


177895 






(0.085) 


(0.025) 


(0.010) 


(0.006) 


(0.004) 


(82767) 




Reading 


0.282*** 


0.145*** 


0.057*** 


0.039*** 


0.029*** 


159490 






(0.083) 


(0.029) 


(0.011) 


(0.007) 


(0.004) 


(75710) 




within-R^ 


0.00 


0.00 


0.00 


0.00 


0.00 






OLSR^ 


0.84 


0.84 


0.84 


0.84 


0.84 





Robust standard errors in parentheses. Errors clustered around school grade level. 
* Significant at 10% confidence level 

** Significant at 5% confidence level 

*** Significant at 1% confidence level 
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Table [C]: Separated samples, year controls, voucher-options coefficients oniy 







25 

(.94 m) 


50 

(2.02 m) 


75 

(3.68 m) 


90 

(5.56 m) 


Whole 

City 


Observations 
( Groups) 


WKCE 


Language Arts 


0.173 


0.154* 


0.065** 


0.050*** 


0.035*** 


84070 






(0.156) 


(0.083) 


(0.027) 


(0.016) 


(0.010) 


(68444) 




Mathematics 


0.104 


0.076** 


0.031** 


0.025*** 


0.017*** 


132781 






(0.106) 


(0.037) 


(0.013) 


(0.008) 


(0.004) 


(90961 




Reading 


0.100 


0.076** 


0.031*** 


0.023*** 


0.015*** 


131814 






(0.076) 


(0.033) 


(0.011) 


(0.007) 


(0.004) 


(90155) 




Social Studies 


0.059 


0.116 


0.046 


0.040** 


0.036*** 


83750 






(0.162) 


(0.083) 


(0.029) 


(0.019) 


(0.012) 


(68139) 




Science 


0.024 


0.071 


0.033 


0.033* 


0.033*** 


84169 






(0.155) 


(0.088) 


(0.029) 


(0.018) 


(0.010) 


(68446) 




nuithin-E} 


0.26 


0.26 


0.26 


0.26 


0.26 






OLSR^ 


0.93 


0.93 


0.93 


0.93 


0.93 




Terra 
















Nova 


Language Arts 


0.189“ 


0.122*** 


0.048*** 


0.035*** 


0.032*** 


159491 






(0.067) 


(0.025) 


(0.009) 


(0.006) 


(0.004) 


(75711) 




Mathematics 


0.047 


0.036 


0.006 


0.008 


0.009** 


177895 






(0.092) 


(0.027) 


(0.011) 


(0.007) 


(0.004) 


(82767) 




Reading 


0.098 


0.075*** 


0.022** 


0.015** 


0.017*** 


159490 






(0.077) 


(0.029) 


(0.011) 


(0.006) 


(0.004) 


(75710) 




’within-E? 


0.03 


0.03 


0.03 


0.03 


0.03 






OLSE^ 


0.85 


0.85 


0.85 


0.85 


0.85 





Robust standard errors in parentheses. Errors clustered around school grade level. 
* Significant at 10% confidence level 

** Significant at 5% confidence level 

*** Significant at 1% confidence level 



The Effect of Milwaukee's Parental Choice Program on Student Achievement in Milwaukee Public Schools 



16 


March 2009 







Table [D]: Separated samples, grade controls, voucher-options coefficients only 







25 

(.94 m) 


50 

(2.02 m) 


75 

(3.68 m) 


90 

(5.56 m) 


Whole 

City 


Observations 
( Groups) 


WKCE 


Language Arts 


0.142 


0.143* 


0.060** 


0.047*** 


0.032*** 


84070 






(0.166) 


(0.086) 


(0.029) 


(0.017) 


(0.011) 


(68444) 




Mathematics 


0.076 


0.076** 


0.031** 


0.026*** 


0.023*** 


132781 






(0.112) 


(0.039) 


(0.014) 


(0.008) 


(0.005) 


(90961 




Reading 


0.076 


0.076** 


0.031*** 


0.024*** 


0.020*** 


131814 






(0.085) 


(0.035) 


(0.012) 


(0.007) 


(0.004) 


(90155) 




Social Studies 


0.033 


0.106 


0.041 


0.037* 


0.037*** 


83750 






(0.171) 


(0.086) 


(0.031) 


(0.020) 


(0.013) 


(68139) 




Science 


-0.036 


0.044 


0.019 


0.023 


0.027** 


84169 






(0.164) 


(0.090) 


(0.030) 


(0.019) 


(0.011) 


(68446) 




within-R} 


0.25 


0.25 


0.25 


0.25 


0.25 






OLSR^ 


0.93 


0.93 


0.93 


0.93 


0.93 




Terra 
















Nova 


Language Arts 


0.046 


0.073*** 


0.022*** 


0.016*** 


0.009*** 


159491 






(0.065) 


(0.024) 


(0.008) 


(0.004) 


(0.003) 


(75711) 




Mathematics 


-0.003 


0.035 


0.009 


0.010** 


0.013*** 


177895 






(0.079) 


(0.023) 


(0.009) 


(0.005) 


(0.002) 


(82767) 




Reading 


0.040 


0.058** 


0.013* 


0.008* 


0.001 


159490 






(0.069) 


(0.024) 


(0.008) 


(0.004) 


(0.002) 


(75710) 




within-R? 


0.03 


0.03 


0.03 


0.03 


0.03 






OLSR^ 


0.85 


0.85 


0.85 


0.85 


0.85 





Robust standard errors in parentheses. Errors clustered around school grade level. 
* Significant at 10% confidence level 

** Significant at 5% confidence level 

*** Significant at 1% confidence level 
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Table [E]: Separate samples, year and grade controls, voucher-options coefficients only 







25 


50 


75 


90 


Whole 


Observations 






(.94 m) 


(2.02 m) 


(3.68 m) 


(5.56 m) 


City 


( Groups) 


WKCE 


Language Arts 


0.143 


0.143* 


0.059** 


0.046*** 


0.032*** 


84070 






(0.164) 


(0.087) 


(0.029) 


(0.017) 


(0.011) 


-68444 




Mathematics 


0.083 


0.072* 


0.029** 


0.024*** 


0.016*** 


132781 






(0.106) 


(0.038) 


(0.014) 


(0.008) 


(0.005) 


(90961 




Reading 


0.074 


0.068** 


0.027** 


0.021*** 


0.013*** 


131814 






(0.077) 


(0.034) 


(0.011) 


(0.007) 


(0.004) 


-90155 




Social Studies 


0.049 


0.114 


0.045 


0.040** 


0.037*** 


83750 






(0.170) 


(0.087) 


(0.031) 


(0.020) 


(0.013) 


-68139 




Science 


-0.015 


0.055 


0.024 


0.027 


0.028** 


84169 






(0.161) 


(0.091) 


(0.031) 


(0.019) 


(0.011) 


-68446 




nuithin-R3 


0.26 


0.26 


0.26 


0.26 


0.26 






OLSR^ 


0.93 


0.93 


0.93 


0.93 


0.93 




Terra 


Language Arts 


0.001 


0.047* 


0.009 


0.007 


0.006** 


159491 


Nova 




(0.066) 


(0.024) 


(0.008) 


(0.005) 


(0.003) 


-75711 




Mathematics 


-0.039 


0.012 


-0.003 


0.002 


0.009*** 


177895 






(0.079) 


(0.024) 


(0.009) 


(0.005) 


(0.002) 


-82767 




Reading 


-0.005 


0.029 


-0.000 


-0.001 


0.002 


159490 






(0.069) 


(0.024) 


(0.008) 


(0.005) 


(0.002) 


-75710 




nuithin-R? 


0.06 


0.06 


0.06 


0.06 


0.06 






OLSR^ 


0.85 


0.85 


0.85 


0.85 


0.85 





Robust standard errors in parentheses. Errors clustered around school grade level. 
* Significant at 10% confidence level 

** Significant at 5% confidence level 

*** Significant at 1% confidence level 
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Table [F]: Effect sizes, no controls, whole city 



Effect Size 



Combined 


Language Arts 


.10 


Sample 


Mathematics 


.08 




Reading 


.10 


WKCE 


Language Arts 


.24 




Mathematics 


.20 




Reading 


.16 




Social Studies 


.40 




Science 


.40 




Language Arts 


.03 


Terra Nova 


Mathematics 


.02 




Reading 


.05 



Effect sizes calculated using within-sample standard deviations on the 
voucher-options variable, which vary based on the test considered. 
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